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ABSTRACT 

An evaluation was conducted of the second year of the K-3 
District Reading Plan (2001-2002) in the Los Angeles Unified School District 
(LAUSD) , California. In September 1999, the district created the reading plan 
requiring the use of specific programs for low-achieving elementary schools. 
The evaluation also focused on alphabetics, reading fluency, and 
comprehension, all of which are important components of both the Open Court 
and Success for All reading plans selected by the school district. Data for 
the 2001-2002 evaluation came from principal (n=36), coach/facilitator 
(n=58), and teacher questionnaires (n=78) , classroom observations, and 
student scores on the Stanford 9 achievement test. The evaluation focused on 
a sample of 46 elementary schools, but school district achievement data were 
also considered. Overall, school-level implementation of these programs 
improved from year 1 to year 2. Principals, facilitators, and teachers 
continued to express the need for more professional development. Teachers 
indicated that principals did not engage frequently in activities supporting 
the program, but that these activities had increased. Principals and coaches 
believed Open Court was most useful with standard English learners; teachers 
thought it was most useful for those reading above grade level. All educators 
thought Success for All was most useful for those reading above grade level. 
Overall achievement findings support the use of Open Court above Success for 
All, but the. differential gains were minimal. An appendix contains a chart of 
effect sizes for programs. (Contains 34 tables and 5 references.) (SLD) 
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Executive Summary 

This report contains the findings from an evaluation of the second year of the K-3 
District Reading Plan (2001-2002). The focus is on both the implementation and the 
impact of both Open Court and Success for All. Where appropriate, comparisons will be 
made to Year 1 (2000-2001). 

In September 1999, the district created the District Reading Plan requiring the use 
of specific reading programs for low-achieving elementary schools. In addition, the 
board action also included professional development and a coaching component as 
critical features of the plan. The primary goal of the district’s new literacy plan is 
“literacy for every student by third grade.” 

In addition to the focus on the plan, the evaluation also focused on alphabetics, 
reading fluency, and comprehension (areas recently identified by the National Reading 
Panel as central to learning how to read). As alphabetics, reading fluency, and 
comprehension are also important components of both Open Court and Success for All, it 
is expected that observers would witness each of these during our classroom 
observations. 

The key evaluation questions addressed during Year 2 were: 

• How well are Open Court and Success for All being implemented at the school 
level (i.e., materials, professional development, support, and student needs)? 

• How well are the teachers implementing the District Reading Plan in the 
classroom (i.e., time spent on reading/language arts, use of program materials, and 
instructional practices)? 

• To what extent is the District Reading Plan affecting achievement? 

The data for the 2001-2002 evaluation has three sources: (1) principal, 

coach/ facilitator, and teacher questionnaires; (2) classroom observations; and (3) student 



test scores on the Stanford 9. Although this evaluation focuses on a sample of 46 
elementary schools in the Los Angeles Unified School District, district achievement data 
will also be presented. 

How well are Open Court and Success for All being implemented at the school 
level? 

Overall, the school-level implementation improved from Year 1 to Year 2. A 
higher percentage of Open Court and Success for All teachers indicated that they 
received their materials compared to 2000-2001. The principals, the coaches/facilitators, 
and the teachers continued to express a need for more professional development. Open 
Court principals need an overview of the components of the program and the coaches 
specifically need more professional development in teaching the writing component, in 
helping English learners, and in their coaching skills. Open Court teachers felt they 
needed more training in teaching writing, conducting Independent Work Time, pacing the 
program, and helping English learners. Many of these same needs were expressed by the 
Success for All staff. 

Although Open Court and Success for All teachers indicated that a large number 
of principals did not frequently engage in activities supporting the program, there was a 
general increase in these activities compared to Year 1. Open Court principals usually 
found the support the coaches provided to be “very useful” to “somewhat useful.” They 
reported the coaches to be “very useful” in distributing the materials (88%) and in 
managing the assessment process (87%). The two categories in which they found the 
coaches to be least useful were facilitating discussions on making accommodations for 
English learners and for students reading above grade level. Open Court teachers did not 
find the coaches as useful as the principals did. They found them “very useful” in 
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distributing or making sure they had all of the instructional materials (71%). Over two- 
thirds found them to be “useful” or “very useful” in distributing materials (89%), 
providing information at small-group meetings (77%), providing information at school- 
wide staff meetings (74%), making sure the classrooms were set up properly (70%), and 
conducting professional development sessions (68%). In every category the Success for 
All principals found the facilitators to be at least “somewhat useful.” The principals 
found them to be most useful in distributing materials (100%). None of the principals 
indicated that support in a category never occurred or that it was not at all useful. Again, 
Success for All teachers did not find the facilitators as useful as the principals did. 

Similar to the principals, the teachers found them most useful in distributing materials 
(91%). However, once “useful” and “very useful” were combined, 80% or more found 
the facilitators to be useful in all of the categories. 

The Open Court principals and coaches believed the program was most successful 
with the standard English learners; however, teachers believed the program to be the most 
successful with students reading above grade level. Overall, all three groups felt the 
program was more effective with their students this year than last year. In fact, all three 
groups felt the program was better in addressing the needs of the English learners in Year 
2 than Year 1. The Success for All principals, facilitators, and teachers all believed the 
program was most successful with the students reading above grade level. Compared to 
last year, principals were less confident regarding the program’s ability to address the 
needs of the English learners but facilitators and teachers felt the same as they did in Year 
1 . 
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How well are the teachers implementing the District Readin£ Plan in the 
classrooms? 

In Open Court schools, all of the grade levels met and exceeded the required 
amount of time on reading/language arts. In fact, teachers were most likely to spend their 
time on reading/language arts followed next by mathematics. As required, most of the 
program materials were displayed in the classrooms; however, the Concept/Question 
Board and the Inquiry area were still not often seen and therefore, not often used. Most 
of the teachers were using the assessment results to guide instruction. Teachers reported, 
and observations confirmed, that they were unable to complete all of the daily lessons. 
Observers indicated that kindergarten teachers were more likely to conduct lessons from 
the Preparing to Read (green) section and grades 1-3 teachers were more likely to 
conduct lessons from the Reading and Responding (red) section. Similar to last year, 
teachers often did not get to the Integrating the Curriculum (blue) section. Some teachers 
were also not conducting Independent Work Time; however, based on a comparison with 
the same second grade teachers, there was an increase from Year 1 to Year 2. 
Observations also showed that many of the teachers were behind on the district’s pacing 
plan for the school year. As teachers mentioned, they needed to make accommodations 
in the program to meet the needs of all of their students. The observers also found that 
alphabetics was most often taught in the early grades but reading for fluency and 
comprehension activities were seen in all four grade levels. Writing was seen in almost 
every classroom; however, it wasn’t always an Open Court lesson. 

The Success for All teachers met and exceeded the required amount of minutes on 
reading/language arts (K-3) and the teachers spent about 90 minutes of this time on 
Success for All activities (Roots and Wings). Observers found that not all of the program 
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materials were displayed and that the teachers couldn’t always finish their lessons. 

Almost all of the teachers were using the assessment results to guide their instruction. 
Alphabetics was seen in all kindergarten and first grade classrooms but few of the second 
and third grade classrooms; however, reading for fluency was seen in most of the 
classrooms and comprehension and writing activities were seen in all of the classrooms. 
The teachers mentioned that they were making accommodations in the program to meet 
the needs of all of their students. The availability of more tutoring slots was specifically 
requested by some of the teachers. 

To what extent is the District Reading Plan affecting achievement? 

Reading program outcomes were assessed by the Stanford 9 Reading, Language, 
and Spelling tests for the entire district and for the subsample of second and third grade 
teachers who were randomly selected for observation. Adjusted NCE gain scores were 
used to compare the impact of the reading programs on students’ reading/language arts 
achievement, taking account of initial differences between the two programs. The overall 
findings support the effectiveness of the Open Court program in comparison to the 
Success for All program; however, the differential gains were minimal. The effect size 
values were either not educationally meaningful or were relatively small. At the district 
level, the only educationally meaningful differences were in Reading Vocabulary (2 nd ), 
Reading Comprehension (3 rd ), and Spelling (3 rd ) in favor of Open Court. 

The goal of the District Reading Plan is literacy for every student by third grade. 
Only 25% of the third grade students were found to be literate based on scoring at the 50 th 
percentile or higher on Reading, Language, and Spelling. However, the percentages are 
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higher in first and second grade, which may reflect the fact that the District Reading Plan 
has been implemented for more years in kindergarten through second grade. 

Based on classroom observations, various indices were constructed as indicators 
of the Open Court 2000 program implementation. Multiple regression analysis was used 
to examine the collective and individual effects of these factors on student gains for 
second grade. Third grade was not included in this analysis since this is the first year that 
the Open Court program was extended to the third grade. For both reading and language 
outcomes, instruction in alphabetics, fluency, and comprehension and instruction in the 
green, red, and blue sections of Open Court were the most significant predictors of 
achievement gains. In addition, adhering to the district pacing plan was another 
significant predictor of language achievement. 
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K-3 DISTRICT READING PLAN EVALUATION 
YEAR 2 REPORT 



I. Introduction 

This report contains the findings from an evaluation of the second year of the K-3 
District Reading Plan (2001 -2002). 1 The focus is on both the implementation and the 
impact of the District Reading Plan as the plan is extended into all grade levels at the 
schools. Where appropriate, comparisons will be made to Year 1 (2000-2001). 

This report consists of four sections. The introductory section presents the 
background of the implementation of the District Reading Plan, descriptions of the two 
most frequently used reading programs (Open Court and Success for All), and the 
evaluation questions. The methods section explains how the sample schools and 
classrooms were chosen and describes the instruments and protocols used. The findings 
section provides answers to the evaluation questions and includes both quantitative and 
qualitative data. Lastly, the fourth section offers conclusions and recommendations. 

A. Background 

During the 1998-1999 school year, the district adopted a new reading/language 

arts policy in which teachers were required to spend at least 90 minutes in kindergarten, 

1 50 minutes in first and second grade, and 120 minutes in third through fifth grade on 

reading/language arts every day. In September 1999, the district expanded this policy 

creating the District Reading Plan for elementary schools. The central component of the 

new plan was that schools scoring below the 50th percentile in reading on the Stanford 

Achievement Tests Ninth Edition in second and/or third grade were required to adopt one 

of three reading programs: Open Court (Collections for Young Scholars or Open Court 

1 During the 1999-2000 school year the plan was only partially implemented in one-quarter of the 
designated schools and therefore the data collected in that year is considered baseline data. 
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2000), Success for All, or Language for Leaming/Reading Mastery. 2 In addition to the 
use of these reading programs, the board action also included professional development 
for staff and a coaching component as critical features of the plan. 

The District Reading Plan was phased in over a two-year period. During Y ear 1 
(2000-2001), Open Court elementary schools were required to implement the program in 
K-2. During Year 2 (2001-2002), Open Court schools were required to introduce the 
program into the remaining grade levels. In contrast, Success for All schools did not 
phase in the program but started using it at all grade levels at the same time. Although 
the schools use the programs in K-5, this report focuses on only K-3 due to the emphasis 
on “learning to read.” By fourth grade, students are expected to be “reading to learn.” 
Furthermore, the primary goal as stated in the district’s new literacy plan is “literacy for 
every student by third grade.” 

Schools not required to use one of the three programs are referred to as Schools 
on Waiver. These schools were exempt from choosing one of the programs because 
students at these schools scored at the 50th percentile or above on second and third grade 
reading or were making substantial gains using another program. Charter schools were 
also exempt from choosing one of these programs. However, some of these schools did 
choose to use Open Court. In addition, students whose parents wanted their children to 
receive primary language instruction were placed in the Waiver to Basic Program. In 
Success for All schools, Waiver to Basic students receive the Success for All program in 
Spanish, but in Open Court or Schools on Waiver, students are taught using a variety of 
different texts and materials. 

2 Only one school chose Language for Leaming/Reading Mastery. As a reasonable interpretation of the 
data cannot be made based on a sample size of one, the school using Language for Leaming/Reading 
Mastery will not be included in the evaluation. 



In a recent report for the Milken Family Foundation on reading programs that 
work, Open Court and Success for All were found to be research-driven and successful 
(Schacter, 2001). However, it is not only important that the programs are based on solid 
research findings, but also that the programs are designed so they can be successfully 
implemented in a wide variety of schools. Therefore, this evaluation focuses on both 
implementation and impact. The implementation evaluation addresses school-level 
implementation (i.e., materials, professional development, support, and student needs) 
and classroom-level implementation (i.e., time spent on reading/language arts, use of 
program materials, and instructional practices). The Stanford Achievement Tests Ninth 
Edition (SAT/9), the primary standardized test used for accountability in California, is 
used to determine the impact of the programs. 

In addition to the focus on specific program instruction, there was a secondary 
evaluation focus on three areas central in learning how to read: alphabetics, reading 
fluency, and comprehension (National Reading Panel, 2000). Alphabetics includes both 
phonemic awareness instruction and phonics instruction whereas reading fluency entails 
both reading speed and accuracy. Fluency strongly relates to our ability to comprehend 
what we read. To practice reading for fluency students read aloud and read silently. 
Comprehension should be the goal in reading, as it is through comprehension that we 
leam and gather information as well as how we get joy out of what we read. As 
alphabetics, reading fluency, and comprehension are important components of both Open 
Court and Success for All, it is expected that observers would witness each of these three 
components in our classroom observations. 
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B. Description of the Reading Programs 

Open Court. During the second year of the District Reading Plan implementation, 
schools were using the following Open Court programs in their classrooms: Collections 
for Young Scholars, Open Court 2000, or a combination of the two. Collections for 
Young Scholars is an earlier edition of Open Court 2000. In 2002-2003, schools using 
Collections for Young Scholars were required to adopt the newest edition of Open Court, 
Open Court 2002. Since Collection for Young Scholars was eliminated, certain analyses 
will focus solely on schools using Open Court 2000. 

The Open Court program emphasizes direct instruction in alphabetic and 
phonological awareness, as well as phonics. In the early grades, the lessons involve 
introduction of 43 common phonemes and their spelling patterns. Students learn these 
sounds through program-specific alphabet sound/spelling cards, alliterative stories, and 
decodable text stories. The shared reading of big books gives way to the reading of 
literature midway though first grade. There are four parts to each Open Court day: 1 ) 
Preparing to Read (referred to as the green section); 2) Reading and Responding (red 
section); 3) Integrating the Curriculum (blue section); and 4) Independent Work Time 
(IWT). Instruction in kindergarten focuses on introducing the alphabet, phonemic 
awareness, letter-sound connections, print awareness, and easy reading. First grade 
expands on kindergarten work to develop reading and writing fluency, build phonemic 
awareness, and learn essential blending and spelling strategies. Second and third grades 
solidify prior learning while teaching the reading skills and strategies that students use to 
gain knowledge and become fluent and automatic readers. According to the program 
developers, an Independent Work Time/Workshop (IWT) should occur every day to 
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allow the students time to practice or extend their learning. This also allows the teacher 
time to work with small groups or individuals accommodating their needs, reteaching 
material, listening to students read, or assessing student progress. As defined by the 
program, alphabetics, reading fluency, and comprehension are all important parts of the 
Open Court program. 

Success for All. Success for All was designed by researchers at Johns Hopkins 
University with the goal of bringing every student to grade level in reading by third 
grade. It was designed for both English and Spanish speaking populations. Both 
phonemic awareness and experience with books are considered critical components of 
this program in the early grades. Through each component of the program, cooperative 
lear nin g is incorporated to provide peer support and modeling. Since Success for All has 
been designed as a proactive, preventative reform model, one-to-one tutoring must be 
provided to identified students at risk. 

In kindergarten, Success for All is a theme-based curriculum that incorporates the 
entire learning time and interweaves through all subject areas (Early Learning). 

Instruction in kindergarten focuses on oral language development and emergent literacy 
skills. Starting in first grade, the Roots program is implemented via the daily 90-minute 
reading block. During this time, students are grouped according to their instructional 
reading level, not their chronological age or grade level. As a result, students often rotate 
daily to a different teacher’s classroom for the homogenous cross-grade Success for All 
lesson. Students are reassessed and regrouped every eight weeks to maintain the 
homogenous groupings. The focus of the Roots program is to have students develop 
sound-blending skills and reading strategies. Students in first grade start with the Roots 
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component of the program and then typically progress to the Wings component in second 
grade; however, depending on their reading performance, children may start Wings 
earlier or later. The focus of the Wings program is to enhance the development and 
application of skills, while continually building fluency, expanding vocabulary, 
deepening comprehension, and fostering thinking skills. Like Open Court, Success for 
All also focuses on alphabetics, reading fluency, and comprehension skills and strategies. 
C. Evaluation Questions 

This evaluation (2001-2002) focuses on program implementation and impact as 
schools phase the programs into additional grade levels at their schools. The key 
evaluation questions that will be addressed this year are: 

1 . How well are Open Court and Success for All being implemented at the 
school level? 

2. How well are the teachers implementing the District Reading Plan in the 
classroom? 

3. To what extent is the District Reading Plan affecting achievement? 

The Los Angeles Unified School District is the second largest school district in 
the United States. To implement a new plan in a district of this size is a monumental task 
that requires several years in order to get all of the pieces in place. Implementation varies 
from school to school and from classroom to classroom. School-level implementation 
consists of material distribution, professional development, support, and student needs. 
Classroom-level implementation focuses on instruction as dictated by program guidelines 
and district policy. Classroom observations present the opportunity to see how much 
time is spent on reading/language arts, if teachers are using the programs, to what extent 
the room is set up according to program guidelines, and most importantly the kinds of 
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instructional activities that are occurring. Academic achievement outcomes will be 
described in relation to the degree of plan implementation. 
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II. Methods 



The data for the 2001-2002 evaluation has three sources: (1) principal, 
coach/facilitator, and teacher questionnaires; (2) classroom observations; and (3) student 
test scores on the SAT/9. Although this evaluation focuses on a sample of elementary 
schools in the Los Angeles Unified School District, district achievement data will also be 
presented. The following sections will describe the sampling technique, the 
questionnaires, the classroom observations, and the academic achievement measures. 

A. Sampling 

A multi-stage probability sample of schools, classrooms/teachers, and students 
was utilized to conduct this evaluation. At the first stage, 50 schools were selected with 
probabilities proportional to student enrollment from a list of all district schools having 
K-3 classrooms, after stratification by administrative region, percentage of English 
learners, and school size. The second stage of selection involved a random selection of a 
single kindergarten, first grade, second grade, and third grade classroom/teacher at each 
sample school for observation. Efforts were made to observe the same teachers from 
year to year; however, some teachers changed positions and/or left the school. These 
teachers were not replaced in the sample. For the first time, third grade teachers were 
included this year as third grade was added to the District Reading Plan in 2001-2002. 
The resultant sample is an equal probability sample of classrooms/teachers districtwide. 

Of the 37 sample schools using Open Court, 28 started in 2000-2001, one school 
started in 1999-2000, and 8 started prior to 1999-2000. 3 Each elementary school 
identified as a District Reading Plan participant was given the opportunity to be an 
Alliance School and implement one of the programs starting in 1999-2000, ahead of the 
3 These figures exclude the Schools on Waiver using Open Court. 
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scheduled district rollout 4 At those schools that volunteered to enter this program, the 
principals received mentoring and leadership professional development for three years. 
Also, schools in the Ten Schools Program had been required to use Collections for 
Young Scholars prior to the implementation of the District Reading Plan. 

Of the six sample schools currently using Success for All, one started the program 
in 2000-2001, three of the schools started the reading program in 1999-2000, and two of 
the schools had prior experience with the program. Of the seven original Schools on 
Waiver schools, four are using Open Court and three are using other reading programs. 
This report will only focus on the 46 schools using Open Court or Success for All. The 
three schools not using Open Court and one school not fully implementing the Open 
Court program in all grade levels were eliminated from the sample. 

As mentioned above, there are two different versions of Open Court being used in 
the district. In 2001-2002 most of the Open Court sample schools (32) used Open Court 
2000, three used Open Court Collections for Young Scholars, and six used both programs 
(typically Collections for Young Scholars in K-2 and Open Court Reading 2000 in grades 
3-5). 

B. Data Collection Methodology 

Principal Questionnaires. In May 2002, each of the 46 principals received a 
questionnaire by school mail. A second letter was sent out as a reminder. After two 
mailings, 37 of the principals responded (80%). Questions focused on whether they 
received their reading program materials, what additional professional development is 
needed, their perceptions regarding the usefulness of the support provided by the coach or 
facilitator, how well the program addressed the needs of their students, and overall 



4 Only one sample school is an Alliance school. 



evaluation of the program. 



Literacy Coach/School Facilitator Questionnaires. In May 2002, each of the 61 
Open Court coaches and the 9 Success for All facilitators received a questionnaire by 
school mail. This year each of the 46 schools was assigned at least one coach or 
facilitator. 5 Of the 46 schools, 21 Open Court schools and 3 Success for All schools had 
co-coaches or co-facilitators. After a letter reminder, 58 (83%) returned their 
questionnaire (52 Open Court coaches and 6 Success for All facilitators). Only one of the 
Open Court schools didn’t have at least one coach that responded (98%) and at least one 
of the facilitators at five of the six Success for All schools responded (83%). Similar to 
the principal questionnaire, questions focused on materials, professional development, 
and program satisfaction. 

Teacher Questionnaires. In spring 2002, all 1 17 of the observed teachers received 
a questionnaire. After two mailings, 67% responded to questionnaires (67 Open Court 
teachers and 1 1 Success for All teachers). Since the sample teachers had been selected at 
random their responses to the questions should be representative of all teachers who 
remained at their same school and taught the same grade level. Questions mirror those 
asked of the principals and coaches/facilitators plus specific questions focusing on the 
support they receive from the coach/facilitator, the support they receive from their 
principal, and their own instructional practices. 

School Observations. Classroom observations are a critical component of this 
evaluation because they provide a window into actual teacher practices. Four classrooms 
(one kindergarten and one first, one second, and one third grade) were randomly selected 
at each school. Over the course of the evaluation, some teachers had left the school, 

5 Last year (2000-01) not all of the Open Court schools had even one coach. 
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changed jobs, or changed grade levels. As indicated earlier, these teachers were not 
replaced; therefore, 117 teachers were observed. Observations occurred unannounced 
between March and early June. The observers spent the entire class time with the 
teachers at all four grade levels to ensure all aspects of reading/language arts instruction 
was observed. Although the visit was unannounced, observations were not conducted, if 
possible, on non-typical days in which activities such as district testing or a field trip 
were planned. 

Trained observers completed a physical-environment checklist, an activity 
summary form, a form indicating how much time was spent on reading/language arts, and 
field notes as part of their classroom observation. There were four checklist forms: 1) 
Open Court; 2) Success for All Early Learning (kindergarten); 3) Success for All Roots 
(grades 1-3); 6 and 4) Success for All Wings (grades 1-3). All of the checklists focused on 
items needed to conduct the program activities. A total score was calculated as an index 
of how well teachers set up their classrooms as specified by program guidelines. One 
item, “Displays student work,” was included on the checklist but was eliminated from the 
total score since it was a general item and not a program item. The remainder of the 
items included both classroom furniture arrangements (i.e., desks facing front or in 
clusters and a class mg area) and classroom displays. The observer completed this form 
while conducting the classroom observation. 

Observers were asked to complete an activity summary form at the conclusion of 
the observation while reviewing their field notes. The summary form included a list of 
different types of classroom activities as well as open-ended questions related to the use 

6 Although the Roots and Wings classes can have students in grades 1-5, the teachers are teaching grades 1, 
2, or 3 in their regular classrooms. 
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of differentiated instruction, integration of reading/language arts into other subjects, and 
the provision of instruction in alphabetics, fluency, and comprehension. On a separate 
form, the observers also indicated the amount of time spent before lunch, after lunch, and 
over the course of the entire day on reading/language arts. 

In order to obtain direct evidence of classroom instruction, observers documented 
teacher-student activities through detailed narratives. The observers noted the time an 
activity began, teacher and student behaviors, the materials used, and the time the activity 
ended. These field notes provide us with information on both the type of activities 
occurring and the amount of time spent. Training on the use of the forms and observation 
techniques were provided to the observers, all of whom had previous experience 
conducting observations either on this project or in other PERB evaluations. 

Coders were hired to review the summary fonns and minute tally forms and to 
complete an additional program summary form focusing on items specific to each 
program. For Open Court, coders specified the section from which they saw instruction 
(i.e., the green, red, or blue section). For Success for All, questions focused on what kind 
of reading/language arts instruction occurred during Success for All time and what 
occurred during the remainder of the day. Finally, for the Open Court classrooms, an 
experienced Open Court teacher read through the field notes to determine the level of 
implementation occurring in the classrooms. 

Academic Achievement Measures. Kindergarten and first grade students are not 
required by the state to take the SAT/9; however, all first grade students do take it. Total 
Reading, Language, and Spelling scores were analyzed as they relate to reading/language 
arts instruction. In addition, the following reading subtest scores (Word Reading, 
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Reading Vocabulary, and Reading Comprehension) were analyzed depending on the 
grade level (see Table 1). When making comparisons it is important to note that not all 
subtests are given in each grade level. 

Table 1. 

Stanford 9 Tests by Grade Level 





Grade 1 


Grade 2 


Grade 3 


Reading 


Yes 


Yes 


Yes 


Word Reading 


Yes 


No 


No 


Reading Vocabulary 


No 


Yes 


Yes 


Reading Comprehension 


Yes 


Yes 


Yes 


Language 


Yes 


Yes 


Yes 


Spelling 


Yes 


Yes 


Yes 



Where possible, all outcomes are disaggregated by student groups to determine 
whether the program is meeting the needs of specific types of students. 
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III. Findings 



1. How well are Open Court and Success for All being implemented at the school 
level? 

This question focuses on the five main areas of school-level implementation 
(based on questionnaire data): 1) materials; 2) professional development; 3) principal 
support; 4) coach/facilitator support; and 5) student needs. The key results are 
summarized below. 

Program Materials 

Principals are required to purchase a complete set of reading materials for each 
teacher and a textbook and workbook for each student; therefore, provisions were made 
so that funding would not be an issue in acquiring them. Once received, the 
coaches/facilitators distribute the materials to the teachers. Most schools received all of 
their materials in a timely fashion. Even though a higher percentage of Open Court and 
Success for All teachers indicated that they received their materials compared to 2000- 
2001, no teacher or student should be without materials. 

Professional Development 

According to central office staff, almost every staff member attended a 
workshop; therefore, this section addresses additional professional development needs. 
Staff discussed their own professional development needs and those of their colleagues. 

Open Court. Twenty-four of the thirty-two principals indicated that they needed 
additional professional development. Open Court components most frequently cited by 
the principals were: writing (6); Independent Work Time (4); and comprehension (3). In 
addition, five principals wanted help addressing the needs of their English learners, three 
mentioned assistance using the assessment results to guide instruction, and three needed 




14 



24 



training on how to do effective classroom observations. Coaches expressed the need for 
the principal to become more knowledgeable about the program in general. The teachers’ 
answers concerning principals’ professional development needs included: unproving 
interpersonal skills, teaching writing skills, knowing the program, working with English 
learners, and understanding the difficulties of the program. 

Forty-two of the fifty-two coaches indicated that they themselves needed more 
professional development especially in the teaching of writing skills (11). Principals 
indicated the coaches needed additional professional development in teaching strategies 
for English learners (6) as well as in coaching strategies such as working effectively with 
teachers and managing job duties (1 1). The teachers’ responses covered a wide variety of 
topics as to what they felt their coaches needed. 

The most common professional development request from the teachers was for 
assistance with the writing component. Sixteen teachers mentioned writing, either alone 
or in combination with some other concern. The second most common concern was 
Independent Work Time (9). Pacing and helping English learners were also mentioned. 
Twenty-eight of the principals indicated that their teachers needed additional professional 
development, especially in working with English learners, teaching comprehension/ 
vocabulary development, conducting IWT, and teaching the writing component. All of 
the coaches reported that the K-3 teachers needed more professional development 
especially in writing (17) and IWT (16). 

Overall, the principals, the coaches, and the teachers feel there is a need for more 
professional development. The principals need an overview of all of the components of 
the program. The coaches specifically need more professional development in teaching 
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the writing component of Open Court, in helping English learners, and in their coaching 
skills. The teachers felt they needed more training in teaching writing, conducting IWT, 
pacing the program, and helping English learners. The principals and the coaches 
supported this belief as well. 

Success for All. Principals wanted more training in the management and use of 
data as well as specific content skill areas (i.e., phonics/phonemic awareness and reading 
comprehension/vocabulary development). Most facilitators said they needed additional 
professional development in how to use the data to help teachers, help with struggling 
readers, writing, and coaching. The Success for All teachers felt they needed training in 
the following areas: meeting the needs of all of their students; grammar; the writing 
component; reading comprehension; implementation of Roots Level 4; transitioning from 
Roots to Wings; and teaching the reading program to English learners. 

Principal Program Support 

Schools are supposed to be “learning communities” where principals, 
coaches/facilitators, teachers, parents, and students all work together with a shared 
commitment to the program and learning. This and the next section focus on what the 
principals and coaches/facilitators do to support reading/language arts in their schools 
and the perceived usefulness of their actions. Findings focus on both principal support 
and coach/facilitator support with respect to their job duties as defined by their job 
descriptions. 

Morris (2002) found that the principal was a key factor in differentiating 
successful schools from struggling schools. For example, at successful schools, 
principals participate in meetings, encourage staff to believe in the program, and visit 
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classrooms. This was not true at struggling schools. Lack of principal support was also 
an area of concern in last year’s evaluation. Consequently, teachers were asked to 
indicate how often their principal engaged in activities related to the reading program or 
reading/language arts in general. 

Open Court. Principal support for the use of the Open Court program was strong, 
largely due to its perceived success at their schools. Compared to Year 1, there was an 
increase in the percentage of principals who supported the program strongly, but a slight 
decrease in overall support of the program (see Table 2). 

Table 2. Open Court Principals (32) 

How do you feel about the primary reading program your school is using? 

2000-2001 2001-2002 

Strongly support 68% 77% 

Generally support 28% 1 6% 

To tal indicating support 96% 93% 

Teachers were asked about the extent to which principals engaged in a variety of 
program activities supporting the program. Although teachers indicated that a large 
number of principals did not engage in these activities or did so only once, there was a 
general increase in these activities compared to Year 1. However, between 12% and 66% 
of the teachers indicated some activities never happened . Table 3 presents the teachers’ 
views of their principals. 
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Table 3. Open Court Teachers (6T> 

How often did your principal engage in the following activities? 





Never 


Once 


Once per 
semester 


Bi- 

monthly 


Monthly 


Bi- 

weekly 


Weekly 


Daily 


Observing you teaching a reading / 
language arts lesson 


17% 


20% 


36% 


9% 


n% 


2% 


6% 


0% 


Providing you with feedback based 
on the observation 


26% 


20% 


42% 


5% 


6% 


2% 


0% 


0% 


Discussing reading/language arts 
assessment results 


37% 


14% 


26% 


11% 


11% 


2% 


0% 


0% 


Discussing individual student 
progress in reading/language arts 


66% 


12% 


12% 


5% 


2% 


0% 


3% 


0% 


Discussing the reading program 
at meetings 


12% 


6% 


21% 


18% 


29% 


6% 


8% 


0% 


Discussing how to make accom- 
modations in the reading program 
for your students 


41% 


11% 


21% 


8% 


11% 


5% 


5% 


0% 


Discussing teaching strategies in 
reading/language arts 


26% 


8% 


21% 


17% 


17% 


8% 


5% 


0% 


Discussing theory or research on 
reading 


29% 


18% 


20% 


14% 


14% 


5% 


2% 


0% 


Discussing classroom management 
strategies 


32% 


9% 


21% 


14% 


12% 


6% 


5% 


2% 



In conclusion, more principals indicated that they supported the use of the 
program than in Year 1 . In addition, the teachers’ felt the principal did more to actually 
support the program than they did previously. Unfortunately, the numbers are still very 
low and many of the principals are still not offering any support in key areas. 

Success for All. Principals were asked both years how they felt about the use of 
Success for All in their schools. There was a definite decline in support for the program 
from 2000-2001 to 2001-2002 (see Table 4). 

Table 4. Success for All Principals 151 

How do you feel about the primary reading program your school is using? 





2000-2001 


2001-2002 


Strongly support 


75% 


60% 


Generally support 


25% 


20% 


Total indicating support 


100% 


80% 



Similar to Open Court, the teachers indicated that many of the principals did not 
engage in activities supporting the program or did so minimally. However, 91% 
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indicated that the principal did observe them once per semester but often without 
feedback (see Table 5). According to the teachers, the principals engaged more in these 
activities this year than they did in Year 1. 



Table 5. Success for All Teachers fill 

How often did vour principal engage in the following 

activities: 





Never 


Once 


Once per 
semester 


Bi- 

monthly 


Monthly 


Bi- 

weekly 


Weekly 


Daily 


Observing you teaching a reading/ 
language arts lesson 


9% 


0% 


91% 


0% 


0% 


0% 


0% 


0% 


Providing you with feedback based 
on the observation 


27% 


9% 


64% 


0% 


0% 


0% 


0% 


0% 


Discussing reading/language arts 
assessment results 


9% 


0% 


64% 


0% 


27% 


0% 


0% 


0% 


Discussing individual student 
progress in reading/language arts 


20% 


20% 


30% 


10% 


20% 


0% 


0% 


0% 


Discussing the reading program 
at meetings 


0% 


0% 


18% 


27% 


36% 


0% 


18% 


0% 


Discussing how to make accom- 
modations in the reading program 
for your students 


18% 


36% 


18% 


9% 


18% 


0% 


0% 


0% 


Discussing teaching strategies in 
reading/language arts 


0% 


18% 


36% 


18% 


18% 


9% 


0% 


0% 


Discussing theory or research on 
reading 


9% 


18% 


36% 


9% 


9% 


9% 


9% 


0% 


Discussing classroom management 
strategies 


9% 


9% 


36% 


0% 


36% 


0% 


9% 


0% 



In conclusion, although Success for All principals indicated that they felt less 



support for the program than they did in the past, there was an increase in supportive 
activities as compared to last year. However, like Open Court, these numbers are still 
very low. 

Coach/Facilitator Program Support 

The coaching component is crucial to the successful implementation of the 
program. The principal sets the tone and vision for the school and the coach/facilitator 
works together with the principal as part of the instructional team. The coach/facilitator 
duties include demonstrating lessons, observing classrooms and providing feedback. 
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ordering and distributing materials, planning and providing staff development, and 
organizing assessments. Coaches/facilitators receive continuous professional 
development on both program content and coaching skills. 

Open Court. Coaches indicated the number one use of their time was time spent 
in the classroom, followed by time spent in meetings and doing paperwork/office work. 
Twenty-three of the coaches (44%) indicated that this was not their first year coaching. 
Ten felt there were positive changes this year, nine indicated both positive and negative 
items, and four provided only negative comments. Positive comments focused on being 
more knowledgeable, working better with the teachers, and having more administrative 
support. Negative comments focused on lack of administrative support and more paper 
work. 

Tables 6 and 7 show how useful principals and teachers found the support 
provided by their coaches. Principals found the support the coaches provided to be 
almost always “very useful” to “somewhat useful.” They reported the coaches to be 
“very useful” in distributing the materials (88%) and in managing the assessment process 
(87%). When combining “useful” and “very useful,” over 60% of the principals found 
the coaches to be supportive in every category. This was especially true for distributing 
the materials (100%), managing the assessment process (100%), conducting 
demonstration lessons (97%), and making sure the classrooms were set up properly 
(94%). The categories in which they found the coaches to be least useful were 
facilitating discussions on making accommodations for English learners as well as for 
students reading above grade level. 
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Table 6. Open Court Principals (32) 

How would you rate the usefulness of the support provided by your school's coach: 





Not at all 
useful 


Somewhat 

useful 


Useful 


Very 

useful 


Did not 
occur 


Conducting demonstration lessons 


0% 


3% 


31% 


66% 


0 


Managing the assessment process 


0% 


0% 


13% 


87% 


0 


Discussing assessment results with teachers 


0% 


10% 


23% 


68% 


1 


Discussing assessment results with you 


3% 


9% 


25% 


63% 


0 


Helping teachers to plan instruction based on assessment information 


0% 


16% 


39% 


45% 


1 


Helping teachers with program pacing 


0% 


13% 


29% 


58% 


0 


Observing the teaching of the program and providing feedback 
Facilitating discussions on making accommodations to meet the needs 


3% 


25% 


13% 


59% 


0 


of English learners 

Facilitating discussions on making accommodations to meet the needs 


3% 


37% 


27% 


33% 


1 


of standard English learners 

Facilitating discussions on making accommodations to meet the needs 


3% 


23% 


33% 


40% 


1 


of students with reading disabilities/difficulties 
Facilitating discussions on making accommodations to meet the needs 


3% 


24% 


38% 


34% 


3 


of students reading above grade level 


3% 


34% 


21% 


41% 


2 


Conducting professional development sessions 

Providing information at small-group meetings (i.e., grade-level or 


0% 


9% 


25% 


66% 


0 


program-level) 


0% 


9% 


22% 


69% 


0 


Providing information at school-wide staff meetings 


0% 


6% 


26% 


68% 


1 


Showing teachers the accommodations video produced by the district 
Distributing or making sure the teachers have all of the instructional 


7% 


.7% 


38% 


48% 


2 


materials 


0% 


0% 


13% 


88% 


0 


Making sure the classroom is set up properly 


0% 


6% 


31% 


63% ' 


0 



The teachers, however, did not find the coaches as useful as the principals did. 



They found them “very useful” in distributing or making sure they had all of the 
instructional materials (71%). Over two-thirds found them to be “useful” or “very 
useful” in distributing materials (89%), providing information at small-group meetings J 
(77%), providing information at school-wide staff meetings (74%), making sure the 
classrooms were set up properly (70%), and conducting professional development 
sessions (68%). Unfortunately, in all but one category some of the teachers indicated that 
these job duties were not even occurring. 
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Table 7. Open Court Teachers (67) 

How would you rate the usefulness of the support you receive from your school's coach: 





Not at all 
useful 


Somewhat 

useful 


Useful 


Very 

useful 


Did not 
occur 


Conducting demonstration lessons 


, 27% 


20% 


27% 


27% 


10 


Discussing the progress of individual students 


27% 


29% 


20% 


25% 


10 


Planning instruction based on student assessment information 


22% 


20% 


33% 


25% 


11 


Helping you with program pacing 


13% 


21% 


28% 


38% 


5 


Observing your teaching of the program and providing feedback 
Facilitating discussions on making accommodations to meet the needs 


24% 


24% 


31% 


21% 


8 


of your English learners 

Facilitating discussions on making accommodations to meet the needs 


25% 


31% 


27% 


16% 


10 


of your standard English learners 

Facilitating discussions on making accommodations to meet the needs 


18% 


27% 


36% 


18% 


9 


of your students with reading disabilities/difficulties 
Facilitating discussions on making accommodations to meet the needs 


24% 


31% 


31% 


15% 


10 


of your students reading above grade level 


27% 


29% 


35% 


8% 


14 


Conducting professional development sessions 

Providing information at small-group meetings (i.e., grade-level or 


10% 


23% 


31% 


37% 


3 


Program- level) 


8% 


15% 


40% 


37% 


1 


Providing information at school-wide staff meetings 


10% 


17% 


37% 


37% 


3 


Showing you the accommodations video produced by the district 


20% 


20% 


28% 


32% 


11 


Distributing or making sure you have all of the instructional materials 


2% 


9% 


18% 


71% 


0 


Making sure your classroom is set up properly 


' 8% 


22% 


30% 


40% 


5 



Teachers were asked to indicate how frequently they received help from the 
coaches in a few key areas. According to the teachers, coaches observed them and gave 
them feedback an average of 3.5 times, with a range of 0 to 15 times. Ten teachers (16%) 
said they were never observed. Teachers indicated that the coach gave demonstration 



lessons an average of 1 .8 times, with a range of 0-12 times. Over one-third of the 
teachers specified that they never saw one. Finally, teachers were asked how often 
coaches helped them and then watched them practice. This was the least likely out of the 
three activities to occur. Teachers indicated an average of .87 times, with a range of 0-5 
times. Over half (57%) said it never occurred. However, it is important to note that the 
district does not require the coaches to work personally with each teacher and that may 
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account for the low numbers of classroom observations and help given to them on 
specific skills areas. 

Success for All. The facilitators were asked how they spent their time. Four of 
the six indicated the assessments and/or regrouping as the activity that accounted for 
most of their time, one said classroom observations, and one said disseminating 
materials. Two facilitators indicated that this was not their first year as a facilitator. In 
contrast, one felt that each year is easier, mentioning the support from the lab staff 
whereas the other one felt that each year is more difficult because they keep adding new 
parts. 

In every category the principal found the facilitator to be at least “somewhat 
useful.” Similar to Open Court, the principals found them to be most useful in 
distributing materials (100%). Eighty percent or more found them to be “useful” or “very 
useful” in all of the categories. None of the principals indicated that support in a 
category never occurred or that it was not at all useful (see Table 8 for the principals 
responses). 
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Table 8. Success for All Principals (5) 

How would vou rate the usefulness of the support provided by your school's facilitator: 





Not at all 
useful 


Somewhat 

useful 


Useful 


Very 

useful 


Did not 
occur 


Conducting demonstration lessons 


0% 


0% 


40% 


60% 


0 


Managing the assessment process 


0% 


20% 


0% 


80% 


0 


Discussing assessment results with teachers 


0% 


20% 


0% 


80% 


0 


Discussing assessment results with you 


0% 


20% 


0% 


80% 


0 


Conducting the tutoring program 


0% 


0% 


40% 


60% 


0 


Helping teachers to plan instruction based on assessment information 


0% 


0% 


40% 


60% 


0 


Helping teachers with program pacing 


0% 


0% 


40% 


60% 


0 


Observing the teaching of the program and providing feedback 
Facilitating discussions on making accommodations to meet the needs 


0% 


0% 


40% 


60% 


0 


of English learners 

Facilitating discussions on making accommodations to meet the needs 


0% 


0% 


60% 


40% 


0 


of standard English learners 

Facilitating discussions on making accommodations to meet the needs 


0% 


0% 


80% 


20% 


0 


of students with reading disabilities/difficulties 
Facilitating discussions on making accommodations to meet the needs 


0% 


20% 


40% 


40% 


0 


of students reading above grade level 


0% 


20% 


40% 


40% 


0 


Conducting professional development sessions 

Providing information at small-group meetings (i.e., grade-level or 


0% 


0% 


40% 


60% 


0 


program-level) 


0% 


20% 


0% 


80% 


0 


Providing information at school-wide staff meetings 
Distributing or making sure the teachers have all of the instructional 


0% 


0% 


20% 


80% 


0 


materials 


0% 


0% 


0% 


100% 


0 


Making sure the classroom is set up properly 


0% 


0% 


20% 


80% 


1 0 



Overall, teachers did not find the facilitators as useful as the principals did. 



Similar to the principals, the teachers found them most useful in distributing materials 
(91%). However, once “useful” and “very useful” were combined, 80% or more found 
the facilitators to be useful in all of the categories (see Table 9). 
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Table 9. Success for All Teachers (11) 

How would you rate the usefulness of the support you receive from your school's coach: 





Not at all 
useful 


Somewhat 

useful 


Useful 


Very 

useful 


Did not 
occur 


Conducting demonstration lessons 


20% 


0% 


40% 


40% 


5 


Discussing the progress of individual students 


9% 


9% 


55% 


27% 


0 


Planning instruction based on student assessment information 


9% 


0% 


64% 


27% 


0 


Helping you with program pacing 


9% 


9% 


46% 


36% 


0 


Observing your teaching of the program and providing feedback 
Facilitating discussions on making accommodations to meet 


9% 


0% 


64% 


27% 


0 


the needs of your English learners 

Facilitating discussions on making accommodations to meet 


10% 


10% 


70% 


10% 


1 


the needs of your standard English learners 
Facilitating discussions on making accommodations to meet 


9% 


0% 


55% 


36% 


0 


the needs of your students with reading disabihties/difficulties 
Facilitating discussions on making accommodations to meet 


9% 


9% 


64% 


18% 


0 


the needs of your students reading above grade level 


10% 


0% 


70% 


20% 


1 


Conducting professional development sessions 

Providing information at small-group meetings (i.e., grade-level or 


0% 


9% 


36% 


55% 


0 


program-level) 


0% 


9% 


55% 


36% 


0 


Providing information at school-wide staff meetings 


0% 


9% 


64% 


27% 


0 


Distributing or making sure you have all of the instructional materials 


0% 


9% 


0% 


91% 


0 


Making sure your classroom is set up properly 


0% 


18% 


45% 


36% 


1 0 



Similar to the Open Court teachers. Success for All teachers indicated that they 



were observed with feedback an average of 3.5 times. One teacher was never observed. 
However, Success for All facilitators rarely gave demonstration lessons (.45 times) or 
helped the teacher and then watched them practice (.64 times). Most indicated that these 
last two activities never occurred. 

Overall, the principals and the teachers found the coaches/facilitators to be useful 
in most areas. However, in both programs, the principals felt they were more useful than 
the teachers found them to be. Although the coaches/facilitators were very useful in 
many areas, they need to move their attention away from materials and set-up to those 
areas that help the teachers’ instructional activities. 
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Student Needs 



Principals, coaches/facilitators, and teachers were asked how well the program 
addressed the needs of different types of students. In Year 1 there were definite opinions 
that the program was not addressing the needs of English learners and students with 



reading disabilities. 

Open Court. The principals and the coaches believed the program was most 
successful with the standard English learners; however, the teachers believed the program 
was most successful with students reading above grade level. This was consistent across 
grade levels. All three groups believed it was least successful with the students who have 
reading disabilities/difficulties. Overall, the three groups believed the program was more 
effective with their students this year them last year. In fact, last year only 35% of the 
principals found it successful with English learners, whereas this year 69% said it was 
successful. More coaches this year also believed the program was helping English 
learners than was the case last year (52% to 64%). The teachers also saw the program as 
more effective for English learners, and, in fact, felt that the needs of all four types of 
students were better addressed. However, of the three groups, the teachers were also the 
most likely to indicate the program wasn’t addressing the needs of some of their students 
(see Tables 10, 11, and 12). 



Table 10. Open Court Principals (32^ 



How well does the readme program address tne instructional needs oi tne ioiiowmg types ui stuueuts; 




Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


English learners 


3% 


28% 


50% 


19% 


Standard English learners 


0% 


10% 


58% 


32% 


Students with reading disabilities/difficulties 


3% 


59% 


28% 


9% 


Students reading above grade level 


3% 


31% 


38% 


28% 
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Table 11, Open Court Coaches (521 

How well does the reading program address the instructional needs of the following types of students? 





Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


English learners 


2% 


34% 


28% 


36% 


Standard English learners 


0% 


4% 


33% 


64% 


Students with reading disabilities/difficulties 


2% 


38% 


46% 


14% 


Students reading above grade level 


0% 


6% 


35% 


60% 



Table 12. Open Court Teachers (67) 

How well does the reading program address the instructional needs of the following typ es of students? 





Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


No 

students 


English learners 


12% 


41% 


31% 


17% 


6 


Standard English learners 


2% 


14% 


53% 


32% 


5 


Students with reading disabilities/difficulties 


26% 


51% 


16% 


7% 


4 


Students reading above grade level 


5% 


15% 


30% 


50% 


4 



In conclusion, principals, coaches, and teachers believed that the program, during 
the second year, was doing a better job of addressing the needs of the students as 
compared to the first year. 



Success for All. The principals, the facilitators, and the teachers all believed 
Success for All was most successful with the students reading above grade level. 
Although none of the principals felt it was not successful with any of the four groups, 
17% of the facilitators and 20% of the teachers felt it was “Not at all successful” with the 
students with reading disabilities. One teacher (9%) also found it unsuccessful with the 
English learners. Compared to last year, principals were less sanguine regarding the 
program’s ability to address the needs of the English learners but facilitators and teachers 
felt the same as they did last year (see Tables 13, 14, and 15). 
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Table 13. Success for All Principals 

How well does the reading program address the instructional needs of the following types of students? 





Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


English learners 


0% 


60% 


40% 


0% 


Standard English learners 


0% 


40% 


60% 


0% 


Students with reading disabilities/difficulties 


0% 


40% 


60% 


0% 


Students reading above grade level 


0% 


20% 


20% 


60% 



Table 14. Success for All Facilitators (6) 

How well does the reading program address the instructional needs of the following types of 
students? 



' 


Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


English learners 


0% 


33% 


33% 


33% 


Standard English learners 


0% 


17% 


33% 


50% 


Students with reading disabilities/difficulties 


17% 


33% 


50% 


0% 


Students reading above grade level 


0% 


0% 


17% 


83% 



Table 15. Success for All Teachers (11) 

How well does the reading program address the instructional needs of the following types of students? 





Not at all 
successful 


Somewhat 

successful 


Successful 


Very 

successful 


No 

students 


English learners 


9% 


55% 


18% 


18% 


0 


Standard English learners 


0% 


70% 


30% 


0% 


1 


Students with reading disabilities/difficulties 


20% 


70% 


10% 


0% 


1 


Students reading above grade level 


0% 


18% 


18% 


64% 


0 



In conclusion, staff felt Success for All was better for those students reading 
above grade level than for English learners, standard English learners, and those with 



reading disabilities. 
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2. How well are the teachers implementing the District Reading Plan in the 
classrooms? 

Classroom-level implementation is the key to the success of this program as 
program fidelity is reported to be critical to student learning. To get the most accurate 
picture of what was occurring in the classrooms, information was gathered in two ways. 
Teachers reported what they did in their classrooms and trained observers documented 
classroom activities. The first two sections address room set up and amount of time spent 
on reading/language arts. The next sections address those aspects of the programs found 
to be problem areas during last year’s evaluation (i.e., using assessments to guide 
instruction, finishing daily lessons, following Open Court program pacing, 
accommodating the lesson to the students, conducting Open Court Independent Work 
Time, and using the Success for All tutoring component). Depending on the program, the 
observational data are broken down by either grade level (Open Court: K-3) or program 
level (Success for All: Early Learning, Roots, and Wings). 

Room Set Up 

According to program guidelines, there are items that should be prominently 
displayed in the classroom and furniture should be arranged in a certain formation. For 
example, in Open Court classrooms, the alphabet/sound cards are to be displayed so the 
students can easily see them. They are used to teach the pupils the sounds and their letter 
correspondence. Desks should be placed so all of the students are facing front. This is 
often a “U” formation. In Success for All classrooms, the cooperative learning standards 
must be displayed as they are; the core of the Success for All program. They are taught 
and practiced in all grade levels. Desks should be arranged in clusters to promote 
cooperative learning. Activities are often done in pairs or small groups. Items were 
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tallied individually and as part of a total score. The Year 1 report indicated that the 
teachers did not have all of the essential physical elements of the programs present in 
their classrooms. 

Open Court. Over 90% of the classrooms displayed student work, had a class 
library, displayed the sound/spelling cards, and had a rug area for the students. The 
display of the word wall increased from last year in K-2, all being over 90%. Similar to 
last year, the use of the Concept/Question Board and the Inquiry area were still not 
always present, although there was an increase from Year 1 . The arrangement of the 
desks improved in all three grade levels (K-2) and third grade showed a similar pattern to 
the other grade levels. The average classroom had 6 of the 8 items. Third grade had the 
highest composite score with an average of 5 out of 6 items present. See Table 16 for the 



findings. 



Table 16. 










Open Court Physical Environment 










Kindergarten 


First Grade 


Second Grade 


Third Grade 


Items 


(n = 24) 


(n = 20) 


(n = 14) 


(n = 36) 


Display of student work 


100% 


95% 


100% 


95% 


Class library 


92% 


95% 


93% 


95% 


Alphabet cards 


100% 


100% 


100% 


97% 


Word wall 


92% 


95% 


93% 


81% 


Concept/question board 


71% 


84% 


86% 


78% 


Inquiry area 


33% 


32% 


29% 


42% 


Floor/rug area 


100% 


100% 


93% 


na 


Big book 


78% 


55% 


57% 


na 


Tables facing front 


61% 


75% 


86% 


81% 


Total (Mean number of items)* 


6 


6 


6 


5 



*N=8 items in K-2. N= 6 items in grade 3. 

Note: Classrooms were eliminated if they were testing, painting, or moving. 



Success for All. Success for All is different than Open Court in that there are 
different items required at each level (Early Learning, Roots, and Wings), although some 
items do overlap (see Table 17). 
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Table 17. 

Success for All - Physical Environment 




Kinder 


Roots 


Wings 


Items 


(n=2) 


(n=2) 


(n=10) 


Display of student work 


100% 


0% 


80% 


Class library 


100% 


100% 


90% 


Tables in clusters/teams 


50% 


100% 


90% 


Alphabet cards posted 


100% 


100% 


na 


Peabody Lang Development Kit/Lang Links 


50% 


50% 


na 


"Practice Active Listening" poster 


50% 


100% 


70% 


"Help and Encourage Each Other" poster 


50% 


100% 


60% 


"Everyone Participates" poster 


50% 


100% 


60% 


"Explain Your Ideas and Tell Why" poster 


50% 


100% 


50% 


"Complete Tasks" poster 


0% 


100% 


60% 


Evidence of a theme 


100% 


na 


na 


Evidence of letter investigation 


100% 


na 


na 


Objects in the room are labeled 


100% 


na 


na 


Environmental print posted 


0% 


na 


na 


"Kinderfriends" posted 


100% 


na 


na 


Dramatic Play Center 


100% 


na 


na 


Construction Center 


100% 


na 


na 


Art Center 


100% 


na 


na 


Library Center 


100% 


na 


na 


Writing Center 


0% 


na 


na 


Evidence of center management 


100% 


na 


na 


Word Wall 


100% 


100% 


78% 


"YES" and "NO" signs 


50% 


100% 


na 


"Word Strategies" poster 


50% 


100% 


80% 


"Bounce and Stretch" poster 


0% 


100% 


na 


"Alphie pennants" above alphabet 


100 


100% 


na 


"STaR" chart 


0% 


100% 


na 


Evidence of a points-reward system 


na 


0% 


80% 


"Alphie" puppet 


na 


0% 


na 


Current letter being studied 


na 


100% 


na 


Team names at individual tables 


na 


0% 


6% 


Green and red words with title of story 


na 


100% 


na 


"Did I Understand What I Read?" poster 


na 


100% 


na 


Word-Mastery List 


na 


na 


90% 


Listening Comprehension Chart 


na 


na 


78% 


Reading Comprehension Objective 


na 


na 


44% 


Day 1-5 guides 


na 


na 


80% 


"Steps in Writing" poster 


na 


na 


90% 


"Meaningful Sentence Checking Strategy" poster 


na 


na 


67% 


Team names posted on wall with rewards 


na 


na 


40% 


Total (Mean number of items)* 






12 



*N=27 items in Early Learning (k). N=22 items in Roots. N=18 items in Wings. Totals were not 
calculated for Early Learning and Roots because of the low number of classrooms. 
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Due to the limited number of classrooms, it would be misleading to make any 
definitive interpretations involving comparisons between Year 1 and Year 2. However, 
similar to Year 1, not everything is displayed as it should be. 

Minutes on Reading/Language Arts 

Teachers are required to spend 90 minutes in kindergarten, 150 minutes in first 
and second grade, and 120 minutes in third grade on reading/language arts every day. 
Since the observers spent the entire day in the classroom, a calculation of the time spent 
on reading/language arts activities was possible. 

Open Court. Last year all three grade levels (K-2) met and exceeded the required 
amount of reading/language arts instruction. In Year 2, all four-grade levels met and 
exceeded the required instructional times (see Table 18). 

Table 18. Open Court 

Average Number of Minutes Spent on Readinq/Lanquaqe Arts 





2000-2001 


2001-2002 


K 


108 


120 


1 


177 


192 


2 


157 


183 


3 


* 


175 



* Third-grade classrooms were not observed in 2000-2001 . 



Only two kindergarten teachers did not teach reading/language arts for 90 minutes 
(82 and 63 minutes out of a total amount of school time of 200 and 190 minutes 
respectively). In first grade, the teachers spent an average of 154 minutes on 
reading/language arts before lunch and an additional 38 minutes after lunch. Only three 
first grade teachers didn’t teach reading/language arts for the required amount of time. 
The second grade teachers spent an average of 159 minutes on reading/language arts 
before lunch and 24 minutes after lunch. Only one second grade teacher did not teach 
reading/language arts for 150 minutes. This was an improvement over last year, where 



five of the same second grade teachers hadn’t taught reading/language arts for the 
required 150 minutes including the one teacher who didn’t this year. Third grade teachers 
spent an average of 144 minutes before lunch and 31 minutes after lunch on 
reading/language arts. 

Success for All. In Year 1, two of the three grade levels met and exceeded the 
required amount of reading/language arts instruction. This year all four grade levels 
exceeded the required amount. Additionally, amount of time spent increased as a 
function of grade level (see Table 19). 

Table 19. Success for All 

Average Number of Minutes Spent on Readina/Lanauaqe Arts 





2000-2001 


2001-2002 


K 


122 


135 


1 


141 


162 


2 


152 


192 


3 


* 


202 



* Third-grade classrooms were not observed in 2000-2001 . 



All but one teacher conducted reading/language arts before and after lunch. The 
time spent during the Success for All block varied from 85 to 93 minutes. Therefore, all 
four grade levels exceeded the required amount of time on reading/language arts and both 
the Roots and Wings classrooms all had approximately 90 minutes of Success for All 
instruction. 

Using the Assessments to Guide Instruction 

Both programs include regular student assessment. Starting during 2001-2002, the 
district set up a web-based system of reporting Open Court school test results and 
generating individual teacher reports. The results are given to the Open Court teachers to 
help them monitor their student’s progress and guide their instruction. Success for All 
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testing is conducted every eight weeks to make sure the students are placed in the 
appropriate Success for All class. 

Open Court. Seventy-eight percent of the K-3 teachers said that they use the 
assessment results. All of the second grade teachers do, and approximately 75% of the 
other teachers said they do. In Year 1, 91% of the K-2 teachers said they used 
assessment results to guide instruction compared to 80% in Year 2. Teachers said that 
they used the test results to determine focal areas for instruction and reviewing. 

Sometimes this is a whole group lesson, but often it is working with individuals or a 
small group on a specific skill. Several teachers mentioned this occurred during IWT. 

Success for All. Almost all of the SFA teachers (91%) said they use the 
assessments. All of the teachers in first and second grade indicated that they do so. This 
was a slight increase from last year (86%). Six commented that they use the assessment 
results to identify weak areas to review and give practice. 

Finishing the Lessons 

Both programs have certain activities that the teachers are supposed to complete 
on a daily basis. In Open Court classrooms, teachers should conduct lessons in the green 
section, the red section, and the blue section as well as conduct IWT and a writing lesson. 
In Success for All classrooms, the activities depend on the program level. 

Open Court. According to the Open Court teachers, in an average week, they are 
able to finish the lessons on 3.8 of the days. Interestingly, the higher the grade level the 
more likely they are able to finish (K: 3.32; 1 st : 3.5; 2 nd : 4.3; 3 rd : 4.17). Both last year and 
this year, 62% indicated that they were unable to complete the lessons every day. This 
was supported by the observations. Teachers were not observed conducting lessons from 
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each of the three sections of Open Court 2000. In kindergarten, teachers were most likely 
to be teaching lessons from the green section; however, in grades 1-3 they were most 
likely to be spending time on red-section lessons. Lessons from the blue section were 
more likely to occur in the upper grade levels (see Table 20). Although writing activities 
were seen in almost all of the classrooms (96% in kindergarten and 100% in grades 1-3), 
they were not always part of the blue Open Court section (see Table 20). 



Table 20. 



Percentage of Teachers Conducting a Lesson in the 
Following Areas 





Preparing to Read 
(Green) 


Reading and 
Responding 
(Red) 


Integrating the 
Curriculum 
(Blue) 


K 


95% 


33% 


43% 


1 


81% 


88% 


44% 


2 


27% 


73% 


64% 


3 


50% 


81% 


61% 



Success for All. Success for All teachers indicated that they were able to finish 
their lessons on 4.45 of the days. Similar to Open Court, the higher the grade level the 
more likely they are able to finish (1 st : 3.5; 2 nd : 4.25; 3 rd : 5). Last year, approximately 
one third of the teachers (36%) indicated that they were able to finish their lessons every 
day. In contrast, 73% of the teachers indicated that they were able to finish them in Year 
2. 

Following the Pacing Plan (Open Court) 

To assist the teachers with pacing, central office staff provided the Open Court 



teachers with a pacing plan that indicated what book and story they should be teaching 
each month. Central office staff indicated that this should be followed within 5 days. 



Using this as our guideline, coders calculated whether each teacher was ahead, on target, 
or behind based on materials used and lessons conducted. In kindergarten, 44% were on 
target, 30% were ahead, and 22% were behind. In first grade, 50% were on target, 32% 
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were behind, and 14% were ahead. 7 In second grade, 44% were on target, 44% were 
behind, and 6% were ahead. Comparing the same teachers in 2000-2001 but using the 
2001-2002 pacing plan, all of the teachers would have been behind in Year 1. About half 
(51%) of the third grade teachers were on target and 44% were behind. If a teacher was 
working on more than one lesson, the lesson farthest along in the books was used in the 
analysis. 

Making Accommodations (Appropriate Strategies for Diverse Learners) 

According to district guidelines, teachers are permitted to make changes in the 
programs to meet their students’ needs. In fact, central office staff provides the Open 
Court coaches with an “accommodations video” to use as a teaching tool with their 
teachers. For Success for All Roots and Wings classrooms there shouldn’t be as much of 
a need for accommodations because the students have already been grouped together 
based on their level; however, it is expected that accommodations may be needed during 
the regular school day when students may be of different levels. Success for All teachers 
are not required to see the “accommodations video.” 

Open Court. Of the 67 teachers that responded to the questionnaire, 1 1 had not 
seen the “accommodations video.” Of those that did, 80% found it to be at least 
“somewhat useful.” 

Last year, almost all of the K-2 teachers (93%) indicated that they made 
accommodations. Similarly, almost all of the K-3 teachers (91%) said that they made 
accommodations in Year 2. However, all of the first and second grade teachers said they 
had made accommodations in Year 2, a slight increase from Year 1. Our observations 



7 Numbers do not always add to 100% because there were a few classrooms in which it could not be 
determined on what lesson they were working. 
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support this finding in that teachers were seen changing the lessons, adding lessons, and 
skipping lessons. Also, as mentioned previously, many of the teachers were not where 
they should be according to the district pacing plan, which may also indicate that they are 
slowing down the lessons or going faster to accommodate their students’ needs. In fact, 
60 of the 67 teachers made reference to accommodating their students. Approximately 
two-thirds of the teachers said they had to make accommodations for their students who 
couldn’t keep up with the lessons. Fifteen teachers specifically mentioned that they had 
to make changes because of their English learners. Eight teachers indicated that they had 
to make the materials more challenging for their high ability students. 

Success for All. Last year, almost all of the K-2 teachers (86%) said they had 
made accommodations. This year, most of the K-3 teachers (80%) said they made 
accommodations. When comparing just the K-2 teachers, there was an increase from 
86% to 1 00%. Third grade was the only grade that had some teachers who didn’t make 
accommodations. Similar to Open Court, the observers did not always see all of the 
Success for All daily components. Two teachers indicated that they had to make changes 
in the pacing or the schedule of the program and two indicated that they gave their 
students extra help and practice. 

Conducting Independent Work Time (Open Court) 

Independent Work Time is a time when students should manage their own work 
time. This is also a time when the teacher can work with small groups and/or individuals 
who need assistance or listen to individual students read. This component of Open Court 
should be occurring on a daily basis. Central office staff provided each school with a 
video showing how an experienced teacher conducts IWT. This video shows how 1TWT 
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changes as the students become more comfortable with the process. By the spring 
semester one would expect to see IWT as defined by the program guidelines. 
Unfortunately, the teachers were not asked if they saw this video. 

According to the Open Court teachers, in an average week, they conduct 
Independent Work Time (IWT) on 3.79 days. This year 43% said they did it on all 5 
days and only 3% indicated that they weren’t doing this part of the program. If they did 
do IWT it usually lasted about 32 minutes with a range of 3 to 90 minutes. There were 
slight differences depending on the grade level. Kindergarten, first, and second all said 
they conducted IWT on four out of five days for 25, 32, and 40 minutes respectively. 
Third grade, which is new to the program in most schools, reported conducting IWT on 
three out of five days for 35 minutes. This finding is similar to last year as 3% also 
indicated that they didn’t do it and those that did also averaged about half an hour a day. 

Classroom observations did not fully support the teacher’s self report data which 
leads to the conclusion that perhaps the teachers do not know how to conduct IWT 
properly or that they are changing it to meet the needs of their students. The observations 
occurred late enough in the school year that we would expect to see a list of “Must Dos” 
and “May Dos” plus the teacher working with individual students or small groups of 
students while this is occurring. For second grade, IWT was observed in 44% of the 
classrooms. It ranged from 27 to 91 minutes with an average of 43 minutes. This was an 
increase from last year when it was only observed in 25% of the second grade 
classrooms. In third grade, IWT was observed in only 24% of the classrooms; however, 
like second grade last year, they are mostly new to the program. The amount of time 
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ranged from 20 to 80 minutes, with an average of 37 minutes. The following is an 
example of a good third grade IWT: 

The students had a “Must Do” list and a “May Do” list to guide their activities. 

Must Do 

1 . Reread current Anthology story 

2. List new vocabulary (find at least seven) 

3. Finish page 69 (2 paragraphs on your invention) 

May Do 

1 . Spider Spelling 

2. Reading 

3. Writer’s Workshop 

4. Word Games 

5. Fluency 

All of these activities were related to Open Court and the current theme and/or story. 
During this time the teacher also worked with two small groups as the other students 
worked on their lists. The teacher conducted a lesson reviewing reading comprehension 
strategies with the first group, using the current Anthology story. With the second group, 
the teacher conducted a lesson on the elements of persuasive writing, during which he/she 
modeled what the students should be thinking about when reading the story. (This was a 
follow-up/extension of a prior lesson.) 

Using the Success for All Tutoring Component 

The Success for All program includes one-to-one daily tutoring for primary 
students who need intervention. The tutoring is designed to reinforce the instruction 
occurring during Success for All time. First priority for tutoring is for 30% of the lowest 
first graders. As financial resources are available, 20% of the lowest second graders and 
10% of the lowest third graders also receive tutoring. 

Only 64% of the Success for All teachers said that they sent their students to the 
Success for All Tutoring Program. Unfortunately, 27% said there was no space available 
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to send the students who needed it. Both first grade teachers sent their students to 
tutoring, three second grade teachers sent them but one indicated there wasn’t enough 
space, and two third grade teachers sent students, two said there was no space and one 
said it wasn’t necessary. Five teachers felt that the tutoring benefited some of their 
students but one felt it did not. One stated, “Yes, I’ve seen tremendous progress.” 

Another felt it helped some of his/her students but that additional tutors are needed so 
more students can receive this intervention. 

Open Court Observation Data 

The following four sections describe what was observed in Open Court 
classrooms. Observation data is separated by grade level. 

Kindergarten. This is the children’s first experience with the Open Court 
program. The primary goal is to introduce the students to literacy. They must learn the 
alphabet and how it works, the connections between sounds with letters, and how sounds 
work together. The teacher must read aloud to the pupils and the pupils must read 
Predecodables and Decodables when they are ready. The teacher should model reading 
to the class emphasizing print awareness as well as comprehension skills and strategies. 
Beginning writing should also be taught here. 

For Open Court 2000 classrooms observers noted whether teachers taught from 
the green section, the red section, and the blue section as well as what they did during 
IWT. There were 21 Open Court 2000 classrooms. Teachers (95%) were most likely to 
be doing lessons from the green section. All but one teacher engaged in preparing to read 
activities. This was not true of the other two sections. During our observations, only 
seven (33%) did activities in the red section and nine (43%) did lessons in the blue 
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section. IWT looked different in different classrooms. In many classrooms it was a time 
for students to go to different centers. In other classrooms, it was a time for students to 
rotate through small groups with the teacher and other adults. 

Based on 27 Collections for Young Scholars and Open Court 2000 classrooms, all 
of the teachers did “alphabetics.” Frequent examples of activities were letter-sound 
correspondence, learning the alphabet (e.g., singing the alphabet song or reciting the 
alphabet), and blending. Twenty-four of the classrooms (89%) spent time reading for 
fluency with reading aloud in unison with the teacher occurring the most frequently. 
Twenty-five of the teachers (93%) spent time on a reading comprehension activity such 
as asking comprehension questions, learning vocabulary words from the story, and 
connecting the story to prior knowledge. Many teachers (70%) were also often seen 
reviewing print awareness during a reading activity. 

Overall, kindergarten teachers switched classroom activities frequently. The 
teacher typically started the day with oral language development such as calendar 
activities, counting, singing, the pledge of allegiance, roll call, and sharing. Teachers 
were observed reviewing the Open Court sound/ spelling cards and often had the students 
sing alphabet songs. Teachers usually focused on one letter/sound, reviewing it and 
conducting follow-up activities like listing words, identifying the sound in words, and 
completing worksheets from the Reading and Writing Workbook. Teachers frequently 
read to the students and modeled and worked on comprehension skills and strategies. 
Students were seen browsing stories and making predictions based on the illustrations. 

At this level, there was not a lot of reading for fluency; however, the students were asked 
to read word lists, individual words or sentences in a story, or echo the teacher reading. 
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As expected, the most common reading activity was the teacher reading aloud to the 
students (78%). Twenty-six classrooms (96%) did some sort of writing. At this level, this 
could be anything from coloring to writing individual letters, words, or strings of words. 
The most common writing activities were drawing/coloring (93%) or working on a 
worksheet (85%). Teachers and students were observed sometimes working on a shared 
writing activity to develop writing skills beyond their present individual writing 
capabilities. Most of the classroom instruction was spent on reading/language arts 
followed by mathematics. School days typically ended with a description of the 
homework, which was usually related to the activities that occurred during class. 

First Grade. The skills taught in kindergarten should be expanded in first grade. 
Phonemic awareness skills should be strengthened and students should be taught 
systematic sound/spelling associations. Teachers must still read aloud but students should 
be developing reading fluency and they should be exposed to authentic literature. They 
also should be developing more fluency in writing, writing sentences instead of just 
letters or words. 

For Open Court 2000 classrooms observers recorded if teachers taught from the 
green section, the red section, and the blue section as well as what they did during IWT. 
There were 16 Open Court 2000 classrooms. Teachers were most likely to be doing 
lessons from the red section (88%), closely followed by the green section (81%). Only 
44% of the teachers did activities from the blue section. Again, IWT varied from centers 
to independent work activities. 

All of the 2 1 teachers did alphabetics, reading for fluency, and reading 
comprehension activities. All of the teachers were seen conducting blending activities. 
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Working on letter-sound correspondence was also seen in most classrooms. Students 
were often seen either reading silently or in unison. In almost every classroom, the 
observers witnessed the students answering comprehension questions (100%) and 
building vocabulary from the story (95%). Other common activities were connecting the 
story to prior knowledge or conducting an activity related to the story. 

Classrooms were seen participating in reading/language arts activities followed by 
mathematics. Even though the teacher still frequently read to the students (71%), the 
students were now seen doing more reading [e.g., reading silently during a lesson (81%), 
reading when they finished an assignment early (81%), taking turns reading aloud (76%), 
reading in unison (67%), and doing Sustained Silent Reading (67%)]. All of the 
classrooms also did some sort of writing. Like kindergarten this was typically 
drawing/coloring and working on a worksheet; however, students were also often seen 
writing in their journals. Oral language activities were common as were reviewing print 
awareness, studying grammar, and working on word definitions and putting the words 
into sentences. Similar to kindergarten, the homework was an extension of the daily 
lessons. 

Second Grade. In second grade there should be a big shift from focusing on 
learning sounds and letters to reading for fluency and comprehension. However, 
previous topics should be reviewed and retaught as necessary. Blending should still be a 
daily routine and Decodables should still be used for practice and building fluency. 

There are 16 Open Court classrooms that were observed in both 2000-2001 and 
2001-2002. Eleven classrooms were using Open Court 2000 in 2001-2002 and ten were 
using it in 2000-2001 . One teacher changed from Collections for Young Scholars to 
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Open Court 2000. In both years, the teachers using Open Court 2000 were more likely to 
do lessons from the red section (82% and 73% respectively). During 2000-2001, the 
green and blue sections were conducted equally (55%); however, during 2001-2002, 
many more teachers conducted lessons from the blue section (64%) as compared to the 
green section (27%). As mentioned in a previous section, there was an increase in 
teachers conducting IWT (25% to 44%) from 2000-2001 to 2001-2002. 

Based on the observations in 16 Collections for Young Scholars and Open Court 
2000 classrooms, all of the teachers had the students read for fluency and all but one had 
the students work on reading comprehension. This was similar for both years. Students 
were seen reading aloud or reading silently. In a few instances fluency tests were 
witnessed. Observers most frequently saw teachers teaching vocabulary from the story 
and asking questions to ensure text understanding (reading comprehension lessons). 
Teachers were also observed teaching a variety of other comprehension skills and 
strategies but not as often. Although less important at this grade level, most also worked 
on some sort of alphabetics, 94% in 2000-2001 and 75% in 2001-2002. 

During both years, reading/language arts was the topic most frequently taught 
followed by mathematics. During 2000-2001, students were often seen taking turns 
reading (75%) or reading silently during a lesson (81%). The following year these two 
activities were also frequently observed (88% and 94% respectively), but so was 
Sustained Silent Reading (69%) or reading when the student finished with a lesson 
(81%). Almost all of the teachers (88%) did some sort of writing activity in 2000-2001 
but they all did writing in 2001-2002. During 2000-2001, this was typically working on a 
worksheet (81%), whereas in 2001-2002 doing worksheets, drawing/coloring, and writing 
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in journals (69% each) were all commonly seen. During both years, the teachers gave 
homework, extending the lessons conducted during class time. 

Third Grade. Third grade should be a continuation of the activities in second 
grade. Again the focus should be on fluency and comprehension. Observations were 
conducted in 39 Open Court classrooms (36 Open Court 2000 and 3 Collections for 
Young Scholars). In the Open Court 2000 classrooms, 50% were seen conducting a 
lesson from the green section. The most commonly observed activity was working on 
building vocabulary. Twenty-nine classrooms (81%) did an activity from the red section. 
As expected, this was typically the students reading aloud and working on 
comprehension skills and strategies. Twenty-two teachers (61%) conducted a lesson from 
the blue section. As previously mentioned, only 24% of the classrooms were seen 
conducting IWT according to the guidelines. 

Based on observations in all 39 classrooms, 59% did alphabetics, 100% read for 
fluency, and 97% conducted comprehension activities. Alphabetics was frequently 
blending or reviewing individual letters. Reading for fluency was typically reading aloud 
or reading silently. Comprehension activities commonly included the teacher asking the 
students comprehension questions or learning vocabulary from the story. Making 
connections to prior knowledge, summarizing, and making predictions were also 
frequently observed. 

Reading/language arts was the most frequently taught subject followed by 
mathematics. Although the teacher was still often seen reading to the students (62%), in 
almost every classroom, the students were seen reading silently during a lesson (95%). 
The students were also frequently seen taking turns reading aloud (74%) or reading when 
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done with their work (69%). Writing was seen in all of the classrooms but the lessons 
were not always Open Court. There were a variety of writing activities observed but the 
most frequently seen were working on a worksheet (85%) or editing (69%). Almost all 
of the classrooms (97%) were also seen working on word definitions. Similar to the other 
grade levels, the students received homework related to their instructional activities that 
day. 

In conclusion, all of the grade levels met and exceeded the required amount of 
time on reading/language arts. In fact, teachers were most likely to spend their time on 
reading/language arts followed by mathematics. As required, most of the program 
materials were displayed in the classrooms; however, the Concept/Question Board and 
the Inquiry area were still not often seen and therefore, not often used. Teachers 
reported, and observers confirmed, that they were unable to complete all of the daily 
lessons and often finished them the next day. In turn, this resulted in them falling behind 
on the district’s pacing plan for the school year. It is important to note that in most 
classrooms the observers saw little wasted time but teachers were still having a hard time 
completing all of the daily lessons. As they mentioned, they needed to make 
accommodations in the program to meet the needs of all of their students. The 
observation uncovered differences in section completion depending on the grade level. 
For instance, in kindergarten teachers were more likely to conduct lessons from the green 
section and in grades 1-3 teachers were more likely to conduct lessons from the red 
section. Similar to last year, teachers often did not get to the blue section. Some teachers 
were also not able to conduct IWT; however, based on a comparison with the same 
second grade teachers, there was an increase from 2000-2001 to 2001-2002. Also, 
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alphabetics was most often taught in the early grades but reading for fluency and 
comprehension activities were seen in all four grade levels. Writing was seen in almost 
every classroom; however, it wasn’t always an Open Court lesson. 

Success for All Observation Data 

The following three sections describe what was observed in Success for All 
classrooms. Unfortunately, there were only two Early Learning and two Roots 
classrooms so generalizability is limited. Descriptions of these classrooms will be 
provided. However, the Wings program was seen in 10 second and third grade 
classrooms so percentages of activities will be presented. 

Early Learning In Kindergarten, the whole day is devoted to the Success for All 
Early Learning Program. The Early Learning Program is a thematically-based integrated 
curriculum that promotes, develops, and enhances oral language, literacy, listening, 
numeracy, creative expression, positive self-esteem, and social skills. The core of the 
program is the five cooperative learning standards: 1. Practice active listening; 2. Help 
and encourage each other; 3. Everyone participates; 4. Explain your ideas and tell why; 
and 5. Complete tasks. These are taught and practiced starting in kindergarten but play 
an important part of every grade level. Their purpose is to create and maintain an 
effective cooperative-learning classroom environment. The Early Learning components 
are: Thematic Units, Phonemic Awareness, Letter Investigations, Emergent Writing, 
Story Telling and Retelling (STaR), Shared Book Experience, Eager to Read, Math, 
Learning Centers, and Peabody Language Development/Language Links. KinderRoots 
begins during the second half of the school year and replaces Eager to Read. The 
following is a brief description of each of the components. There are 10 science and 
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social studies themes that were selected so students can learn to make connections 
between what they leam and the world around them. Each theme usually takes three to 
four weeks to complete. The learning of concepts and subject areas around each theme 
begins at the students’ level of familiarity and then progresses to more complex learning 
as the students’ progress through the theme. Students receive lessons in phonemic 
awareness, letter investigation, and writing as part of their day. Students are taught how 
to recognize sounds in words and to be able to manipulate them (Phonemic Awareness) 
and that the alphabet as well as print is all around us (Letter Investigation). Learning 
how to write is a developmental process starting with scribbling and drawing and 
progressing to writing. The goals of writing in the Early Learning Program are to 
motivate and promote writing, to engage students in the writing process, and to convey to 
the students that they are all writers. Three specific Success for All activities center on 
reading. STaR introduces students to age appropriate literature. It enhances oral 
language, increases vocabulary and comprehension development, and promotes love for 
reading. Shared Book Experience teaches children about literacy, concepts of print 
through use of big books/enlarged text with predictable story lines, and provides time for 
independent reading. Eager to Read is a series of 3 1 simple, theme-related stories, each 
with a big book that develops print awareness and encourages reading-like behaviors. 
Math lessons should be integrated with the theme. It may be part of small/large group 
lessons. Learning Centers, or the calendar activity because of time limitations. Math 
lessons reinforce the theme learning essentials in each Early Learning thematic unit. 
However, it is not a comprehensive math curriculum and therefore teachers must 
supplement these lessons with their appropriate math curriculum. Students should spend 




48 



58 



time at Learning Centers every day for 30 to 45 minutes, working independently and 
interacting with other students. There should be a minimum of seven centers and the 
students should have some choice in the activities they do. The center tasks should be 
related to the theme and include a writing activity. The final piece is Peabody Language 
Development or Language Links, which promotes oral language and cognitive 
development. Each of these activities should occur every day. There is also time in the 

r 

class schedule to conduct lessons not covered above (Specials). 

Both classrooms were observed implementing the Success for All program. In 
one classroom the theme was dinosaurs and in the other classroom the observer was 
unable to determine the theme. Both classrooms did blending and one did Letter 
Investigation. Both had writing activities that included both drawing/coloring and 
sentence writing. For example, one teacher had the class working simultaneously on 
three different levels of writing. The first group colored in a picture of a queen, cut out 
the pieces, and glued them to construction paper. The second group colored in a picture 
of a butterfly and made up or recited a rhyme about butterflies. The third group colored in 
the word blue (hollow letters) and wrote two sentences with the word blue in it. This 
shows three levels of writing activities based on student development. 

Both teachers read to the students and asked them questions. Neither classroom 
had center time, although one of the classrooms had different classroom activities to do 
during recess that was similar to centers. In both rooms the only math related activity 
was part of the calendar activity. Oral language development was an important part of 
instruction in both classrooms and could be witnessed throughout the day. One class 
spent a lot of time on non-Success for All activities (e.g., Sustained Silent Reading, 
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Physical Education, and working on a Mother’s Day present). The other teacher spent 
only a few minutes explaining how to read a clock, prompted by the students’ inability to 
tell the teacher what time it was. 

Both teachers did alphabetics, reading for fluency, and reading for comprehension 
with the students but to different degrees. One did only blending but the other did several 
alphabetic activities centering on the letter “Q,” blending, and rhyming. One teacher 
started the day with eight minutes of reading for fluency (Sustained Silent Reading). The 
observer felt some children seemed to be actually reading whereas others just looked at 
the illustrations. The students read aloud from a book echoing the teacher, read a poem, 
and read their spelling words in unison. The second teacher had the students read the 
morning message and sentences from the board but no literature was read. Listening for 
comprehension was prominent in both classes. One teacher read to the students and 
checked comprehension on two occasions. The other teacher started the lesson by 
activating prior knowledge about dinosaurs before reading them a book and asking them 
questions. Later this same teacher reviewed the author and illustrator, discussed 
vocabulary, and asked the students questions from another story. And in a third story, 
children listened to a tape while the teacher showed them pictures and asked them 
questions. To summarize, one teacher focused primarily on alphabetics and fluency 
while the other focused more on comprehension. 

Roots. For 90 minutes Success for All students attend a Reading Roots class. 
This may be with their homeroom teacher or with another teacher at the school. The 
classroom may be composed of the same grade level students or a mixture of different 
grade levels. The 90-minute class should consist of three sections: STaR (Story Telling 
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and Retelling); Reading Roots; and Peabody Language Development/Language Links. 

The teacher starts with STaR for 20 minutes followed by a Reading Roots Shared Story 
Lesson for 50 minutes, and then oral language for the remaining 20 minutes. STaR is an 
interactive story-reading program. Students build oral language skills through the 
reading, discussing, and retelling of good literature. The focus of STaR is 
comprehension. Reading Roots consists of the following sections: Showtime, Letter 
Activity, Story Activity, and Celebration. On the first day of a cycle of lessons, there is 
also an activity called Setting the Stage. The goals of Reading Roots are to develop 
independent readers, to offer opportunities to practice reading, and to teach decoding and 
comprehension skills. According to the program guidelines, the goal of Peabody 
Language Development/ Language Links is to . .stimulate oral language, writing, and 
cognitive development.” 

Both first grade classrooms were observed implementing Reading Roots. One 
classroom had only 10 students with a mixture of first, third, and fourth grade students. 
The third and fourth grade students were identified as ELD Is (the lowest level). The 
second classroom had 20 first graders and one student who was a second-grade retained 
student. At both schools the students start out with their regular teacher, attend the 90- 
minute Success for All lesson, and then return to their regular teacher. Both teachers 
started with the STaR lesson and then moved into Reading Roots. The observers did not 
see evidence of the third component in either class. Both ended with Celebrations (i.e., 
having a student read aloud or having students read the word wall). However, oral 
language skills were practiced throughout the first two components. Only one class had a 
writing activity during Success for All, a worksheet related to the story they were 
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reading. While both teachers seemed constantly on task during Success for All time, they 
were unable to complete all of the sections. 

Both teachers spent a considerable amount of additional time on other 
reading/language arts activities (84 minutes and 73 minutes). Students practiced writing 
during this time. There were several examples of writing activities but no actual writing 
instruction. Students read stories they wrote for homework aloud and they took spelling 
tests. Observers also witnessed reading/language arts integrated into both science and 
math lessons. 

Both teachers did alphabetics as part of Success for All. This was typically 
studying one letter byreviewing the sound, learning how to write it, and identifying what 
words contain it. The observers witnessed several different approaches to reading for 
fluency. For instance, students engaged in partner reading, students read a list of words 
from the story, students did Sustained Silent Reading, and students read a familiar story 
in unison. Students in both classrooms read literature as well as other kinds of text. They 
read individually and in unison. One teacher asked comprehension questions and built 
vocabulary from the story. The second teacher worked on comprehension questions, 
built vocabulary, had the students make predictions, and activated the students’ prior 
knowledge. 

Wings. All of the second and third grade teachers taught Wings during their 
Success for All class time. Similar to Roots, this is a 90-minute class. Once students 
have completed the Roots lessons successfully they move on to Wings. Usually students 
start Wings at the beginning of second grade; however, this varies depending on student 
progress. Wings builds on the skills learned in Roots. The 90-minute period should 
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consist of three components: Listening Comprehension (20 minutes), Reading Together 
(55 minutes), and Book Club/ Additional Skills Instruction (15 minutes). During 
Listening Comprehension, teachers read literature to students from different genres. 
Listening Comprehension provides opportunities for students to understand the nature of 
stories, various genres of literature, and the use of literary devices in an interactive format 
that models strategic reading. Reading Together is broken down further into Story- 
Related Activities, Direct Instruction in Reading Comprehension, and Two-Minute Edit. 
It includes reading for fluency, vocabulary, discussions, direct instruction in 
comprehension, and writing. The Two-Minute Edit consists of sentences or a short 
paragraph containing errors that the teacher writes or selects. This is a cooperative, 
small-group activity where students edit and explain why the editing is needed. Two or 
three days a week the Success for All session closes with Book Club. This can be 
Sustained Silent Reading or a time to share books the students read at home. On the 
remaining days, this time is used for additional instruction based on students’ needs. 

The Success for All classes consisted of 16 to 23 students. In all classrooms but 
one, it included three to four grade levels. Listening Comprehension was observed in 
90% of the classrooms and Reading Together was observed in all of the classrooms. The 
one classroom that did not do Listening Comprehension spent time on class rules since it 
was the first day of a new group of Success for All students. Book Club was not seen in 
every classroom but this is not unexpected since it is not supposed to occur every day. 
Class time typically consisted of whole group instruction, partner reading/problem 
solving, and individual work. 



ERIC 



53 



63 



Alphabetics was only observed in two classrooms (20%) and it did not occur 
during Success for All time. These were actually not lessons but rather explanations of 
how to sound out a word during reading. One would not expect to see alphabetic lessons 
as these classes had students from the highest Success for All grouping. Reading for 
fluency was seen in most of the classrooms whereas comprehension activities and writing 
were seen in all classrooms. Students were frequently observed reading silently or 
reading with a partner. In every classroom the teacher was seen asking the students 
comprehension questions. In 80% of the classrooms they worked on vocabulary related 
to the story and/or did writing related to the story. In over half of the classrooms they 
related the story to other stories they had read or to prior knowledge, they previewed the 
story and made predictions, and/or they sequenced the events of the story. Lessons often 
incorporated several comprehension skills or strategies (i.e., main idea, relate to own 
experiences, and compare and contrast); however, the teachers usually didn’t ask for any 
higher level of thinking or explanation of answers. 

The most common type of writing was completing a worksheet, such as the 
Treasure Hunt activity. Writing also occurred during non-Success for All time and 
included writing a thank you letter, editing one’s own work, journal writing, and tests. 
Although reading/language arts was not usually integrated into other subjects except at a 
minimal level (e.g., reading or writing), there was one good example of an integrated 
lesson. The following is an example of reading/language arts integrated into Social 
Studies: 

Students were asked to read silently from their Social Studies book about 
conservation. Before having the students alternate reading aloud, the teacher asks 
them what reading strategies they will use. One student responds, “Finding the 
main idea.” Before answering the questions at the end of the chapter the teacher 
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asks the students what they can do. The student’s answer: read the book again, 
look for the answer in the book, or ask someone else. The teacher explains that 
one of the questions will take critical thinking skills. The students do the work 
and the teacher walks around monitoring progress. Before discussing answers, 
the teacher goes over vocabulary from the chapter. She/he reminds the class that 
if they come across a word that they don’t know they can try to figure it out by 
reading the sentences before it and after it. The teacher and students go over the 
answers with the teacher writing one student’s answer on the board. The teacher 
comments that he/she is pleased that the student gave a complete answer and used 
the question to form her answer. The teacher ends the lesson with an extension of 
this lesson on conservation. Students must write down four things that trees are 
used for and illustrate each. 

In conclusion, the teachers met and exceeded the required amount of minutes on 
reading/language arts (K-3) and the teachers spent about 90 minutes of this time on 
Success for All activities (Roots and Wings). Similar to Open Court, not all of the 
program materials were displayed and the teachers couldn’t always finish their lessons. 
The Success for All teachers mentioned that they would like more spaces available for 
their students in the tutoring part of the program. Alphabetics was seen in all 
kindergarten and first grade classrooms but few of the second and third grade classrooms. 
However, reading for fluency was seen in most of the classrooms and comprehension and 
writing activities were seen in all of the classrooms. Similar to Open Court, the teachers 
were usually not seen wasting instructional time but were unable to complete all of the 
Success for All lessons. They also mentioned that they made accommodations in the 
program to meet the needs of all of their students. 
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3. To what extent is the District Reading Plan affecting achievement? 



In this section reading program outcomes are assessed by Stanford. Achievement 
Tests Ninth Edition (SAT/9) for the entire district and for the subsample of teachers who 
were randomly selected for observation. Three tests (Reading, Language, and Spelling) 
and reading subtests (Word Reading, Reading Vocabulary, 8 and Reading 
Comprehension) were included in these analyses. For this evaluation, adjusted NCE 9 
gain scores (adjusting for pre-existing difference among students) provide a meaningful 
measure of academic progress. 

Both student variables [such as ethnicity, language status, and English Language 
Development (ELD) level] and teacher variables (such as years of service and teacher 
credential status) were analyzed in relation to students’ outcomes. 

Finally, for a subset of our sample using the Open Court reading program, the 
impact of instructional variables and program implementation on student outcomes were 
examined. It was not possible to explore the impact of classroom and instructional 
program on student’ outcomes for Success for All students due to the high frequency of 
students’ rotation with different teachers during the school year. 

Evaluation Questions 

The general research question: ‘To what extent is the District Reading Plan 
affecting achievement? ” is translated to the following specific evaluation questions: 

1 . How do Open Court program students achieve compared to Success for All 

program students? 

8 At the first grade level it is Word Reading and at the second and third grade levels it is Reading 
Vocabulary. 

9 NCE score stands for Normal Curve Equivalent score, which has a normal distribution with a mean of 50 
and a standard deviation of 21 (Stanford Achievement Test Series, Ninth Edition, Technical Manual). 
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2. What is the impact of Open Court and Success for All on student achievement? 

3. What percentage of each group of students is literate? (The criterion for 
considering a student literate was scoring at the 50 th percentile or higher on 
Reading, Language, and Spelling as measured by SAT/9.) 

4. Do Open Court and Success for All have the same impact for different ethnic 
groups? 

5. Do Open Court and Success for All have the same impact for students with 
different language classifications? 

6. Do Open Court and Success for All have the same impact for students with 
different ELD levels? 

7. Does teacher experience impact student achievement? 

8. Does teacher permanency impact student achievement? 

9. What are the Open Court implementation factors that impact student outcomes? 

Findings 

Data to answer the above evaluation questions were obtained from three main 
sources: Student Information System (SIS), the district’s employee database, and 
classroom observations. In the following sections of this report each specific question 
will be answered respectively. 
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How do Open Court program students achieve compared to Success for All 
program students? 

2000-2001 Achievement Data . Tables 21 and 22 present the year 2000-2001 
achievement data for Open Court and Success for All reading programs by grade level. 
The SAT/9 was given in Spring 2001, after a majority of students had been exposed to 
the reading programs for at least six months. (Some of these students may have benefited 
from these programs for a longer period of time). However, since we do not have any 
data at the beginning of the program we cannot attribute student achievement to their 
respective reading programs alone. Individual differences related to reading abilities at 
the beginning of the program, are also important elements affecting student progress. 
Measures selected for reading/language arts achievement are Reading, Reading Subtests, 
Language, and Spelling. The major findings drawn from these tables are: 

• 2000-2001 first and second grade Open Court program students had higher 
test scores than Success for All students in all subject areas when analyzed for 
the district and sample teachers. 

• Sample teachers’ average scores are very close to the district average with 
minor variation. This finding validates our sampling process and allows us to 
generalize from sample data to the whole district. 

Overall, Open Court students scored higher than Success for All students across grades 
(first and second), all tests (Reading, Word Reading, Reading Vocabulary, Reading 
Comprehension, Language, and Spelling), and both groups (district and sample teachers). 
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Table 21. 

First Grade Spring 2001 NCE Scores 



Population/ 

Sample 


Subject Tested 


Open Court 


Success for All 


District 




N 


Mean 


SD 


N 


Mean 


SD 


Reading 


48,329 


53.41 


19.73 


4,219 


42.46 


17.92 


Word Reading 


50,075 


54.46 


21.09 


4,531 


43.62 


20.32 


Reading 

Comprehension 


49,462 


51.84 


18.73 


4,403 


42.88 


17.11 


Language 


48,355 


47.50 


19.97 


4,299 


40.40 


17.69 


Spelling 


49,879 


53.95 


24.03 


4,523 


39.00 


20.54 


Sample 

Teachers 


Reading 


264 


51.69 


18.85 


66 


45.29 


17.63 


Word Reading 


273 


54.03 


19.99 


70 


46.29 


21.09 


Reading 

Comprehension 


265 


50.58 


17.85 


70 


44.46 


16.31 


Language 


268 


44.63 


18.59 


70 


39.35 


17.55 


Spelling 


273 


53.12 


22.63 


72 


40.31 


20.34 
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Table 22. 

Second Grade Spring 2001 NCE Scores 



Population/ 

Sample 


Subject Tested 


Open Court 


Success for All 


District 




N 


Mean 


SD 


N 


Mean 


SD 


Reading 


53,627 


43.11 


18.00 


4,806 


35.22 


16.04 


Reading 

Vocabulary 


54,791 


41.55 


22.27 


4,977 


32.77 


20.83 


Reading 

Comprehension 


56,264 


42.08 


16.77 


5,215 


36.36 


15.53 


Language 


55,422 


44.02 


22.37 


5,107 


36.42 


20.50 


Spelling 


55,995 


47.72 


20.94 


5,190 


38.91 


18.72 


Sample 

Teachers 


Reading 


577 


46.20 


16.76 


86 


35.66 


14.55 


Reading 

Vocabulary 


585 


45.42 


19.54 


92 


33.20 


18.63 


Reading 

Comprehension 


604 


43.52 


15.63 


95 


37.85 


15.23 


Language 


593 


47.98 


21.25 


94 


36.30 


18.55 


Spelling 


602 


51.10 


18.38 


95 


39.47 


16.63 
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